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Amendments to the Specification : 

Please replace the paragraph beginning on page 2, line 2, and ending on page 2, line 7, 
with the following paragraph: " 

—The performance of an image processing system can be m e asur e s measured 
in many ways. One measure is "resolution," or the ability to distinguish different 
scene objects or materials in the processed image. "High resolution" means the 
processed image shows different objects that are closely spaced in the scene. Such 
"high resolution" is more properly termed "high spatial resolution." The term "high 
spectral resolution" refers to the system's ability to distinguish some elements that are 
closely related in electromagnetic wavelength.— 

Please replace the paragraph beginning on page 3, line 7, and ending on page 3, line 12, 
with the following paragraph: 

—Another prior art technique is the Projective Pan Sharpening Algorithm 
(published by John Lindgren and Steven Kilston as paper #2818-26, SPDE '96, Denver, 
Colorado). This technique assumes nearly complete spectral overlap between a 
panchromatic band and [[that]] the multi-spectral bands it is used to sharpen. It also 
assumes a linear relationship between intensities in the pan and multi-spectral bands.-- 

Please replace the paragraph beginning on page 6, line 20, and ending on page 7, line 11, 
with the following paragraph: 

—The high-resolution camera or sensor 30 may capture a single or multiple 
spectral bans. In on embodiment, the high resolution sensor 30 is a high-resolution 
panchromatic sensor, which outputs a single intensity that represents an aggregation of 
a wide range of spectral bands. Alternatively, a sensor which captures and outputs 
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either a single or multiple narrow spectral band(s) may be used. Typically, the high- 
resolution camera or sensor configured to acquire all of the spectral bands of interest at 
the same, high resolution. The high resolution camera or sensor 30 outputs an N 
dim e nsional N-dimensional auxiliary image 34 of the field of view to the image 
processor 10. For the auxiliary image 34, N is typically 1 and typically represents the 
panchromatic band. However, N may be greater than one and the spectral bands may 
be any convenient band or bands within or outside of the visible spectrum. Only one 
auxiliary band must have high resolution. The other auxiliary image bans can have any 
resolution equal to or better than that of the source image.— 

Please replace the paragraph beginning on page 7, line 12, and ending on page 7, line 22, 
with the following paragraph: 

-The low resolution camera or sensor 32 may be configured to capture multiple 
spectral bands of interest of a field of view that is co-registered with the high-resolution 
camera 30. The low resolution camera 32 outputs an N dim e nsional N-dimensionah 
low resolution source image 36 of the field of view to the image processor 10. From a 
group of co-registered images of various resolutions, one is chosen as the "source 
image/'ft,]] i.e. the one to be enhanced, one is chosen as the primary auxiliary image 
and must have a higher spatial resolution than the source. Any or all of the images can 
be included as auxiliary images and will serve to improve a spectral discrimination for 
each pixel of the source. Preferably, the spatial resolution of the auxiliary image is an 
integer multiple greater than the source image in any and/or all dimensions. — 
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Please replace the paragraph beginning on page 8, line 19, and ending on page 9, line 3, 
with the following paragraph: 

-First, the method will be described using a single auxiliary image. In step 
100, the image processor 10 acquires the source image 36 and auxiliary image imag e s 
36 and 34 , r e sp e ctiv e ly . The acquired source image 36 has a lower spatial resolution 
than the acquired auxiliary image 34. For this reason, the source image 36 has fewer, 
larger pixels than the auxiliary image 34 as shown in Fig. 4. Additional auxiliary 
images [[3-5]] 35 may also be acquired but are not used in this illustration. The image 
processor 10 may acquire the images 34 and 36 from the camera 12, the database 14, or 
the network 16.— 

Please replace the paragraph beginning on page 9, line 4, and ending on page 9, line 20, 
with the following paragraph: 

-In step 102, the image processor 10 re-samples all of the high resolution 
pixels 52 of the auxiliary image 34 to have the same spatial resolution [[of]] as the 
source image 36. This [[is]] step is shown graphically in Fig. 4. In the simplest case, 
the auxiliary image 34 has a resolution that is an integer multiple of the source image 
36 in any single dimension or in all dimensions. In this case, the image processor 10 
defines clusters 50 within the high resolution pixels 52 based on the integer multiple of 
each linear dimension. For example, if the auxiliary image 34 has twice as many 
pixels in each linear dimension, then each group 50 will include a 2x2 array of high 
resolution pixels 52. Similarly, if the auxiliary image 34 has three times as many 
pixels in each linear dimension, [[than]] then each group 50 will include a 3x3 array of 
high resolution pixels 52. For each group 50, the image processor 10 forms at least 
one average coarse pixel [[54]] 55. The average coarse pixel(s) 55 for each group 50 
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is equal to the average of spectral intensities for the high resolution pixels 52 in each 
group 50. More sophisticated and accurate sub-sampling techniques known in the 
literature (see, for example, Digital Image Processing . W.K. Pratt, John Wiley & Sons, 
New York, 1978) could be used instead of simple averaging.-- 

Please replace the paragraph beginning on page 9, line 21, and ending on page 10, line 2, 
with the following paragraph: 

—The set of coarse pixels 5 4 and 55 [[is]] are then used as the resampled pixel 
values that comprise the re-sampled auxiliary image 38- The coarse pixels 54 in the 
source image 36 and the cours e coarse pixels 55 in the auxiliary image 34 have the 
same spatial resolution after step 102. This is illustrated in the following examples.-- 

Please replace the paragraph beginning on page 10, line 22, and ending on page 11, line 8, 
with the following paragraph: 

—Because the spatial resolution of the auxiliary image is not an integer 
multiple of the spatial resolution of the source image, the auxiliary image is resampled 
twice. First, the auxiliary image 34 is resampled to an intermediate spatial resolution 
of 600x600 pixels. This resampling is done by converting eachl.33 pixels in the 
800x800 pixel auxiliary image 34 to 1 pixel using pixel interpolation. Subsequently, 
the intermediate auxiliary image [[34]] is resampled a second time by grouping high 
resolution pixels 52 in the auxiliary image [[34]] into 2x2 groups 50. The average 
pixel intensity value of each 2x2 group 50 defines a new, coarse pixel 55 representing 
each group in the 300x300 pixel resampled auxiliary image 38. There is no need to 
resample the source image 36 because the resampled auxiliary image 38 is at the same 
spatial resolution as the source image 36.-- 
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Please replace the paragraph beginning on page 13, line 16, and ending on page 13, line 
21, with the following paragraph: 

-The imaging data in step 1 10 is generally comprised of an array of pixels, 
where each pixel represents a position on a source image [[34]] 36 or an auxiliary 
image [[36]]34 and a spectral profile of the position. The spectral profile is represented 
by an N-dimensional vector having N spectral intensities. For many application, N is 3 
and each of the three spectral intensities represents a respective intensity in the visible 
light portion of the spectrum, such as red, green or blue color. -- 

Please replace the paragraph beginning on page 7, line 12, and ending on page 7, line 22, 
with the following paragraph: 

—The auxiliary image 34, like the source image 36, may also have N 
dim e nsional N-dimensional pixel data where N may be 1 , 3, 1 1 , 224 or any other 
convenient number. However, at least one band of the auxiliary image 34 must have 
higher resolution than the source image to permit spectral sharpening according to the 
present invention.-- 

Please replace the paragraph beginning on page 15, line 1, and ending on page 15, line 1 1, 
with the following paragraph: 

—Also, for any given co-located pixel in the source image 36 and the auxiliary 
image 34, a gain vector describes the relationship between differential changes in pixel 
intensities of the auxiliary image to differential changes in pixel intensities of the 
source image. The gain which is determined based on steps 106 and 108 as described, 
is typically a ratio of differential source pixel intensity to differential auxiliary image 
pixel intensity. For each pixel, one ratio is typically formed for each spectral band in 



09/694,782 

the source image as a ratio to each spectral band in the auxiliary image [[24]] 34 that 
has a higher spatial resolution than the source image. Each gain is generally only valid 
for pixels that have imaged vectors that are close to the imaged vector of the 
underlying pixels in the source image 36 and auxiliary image 34 for which the gain 
was calculated.— 

Please replace the paragraph beginning on page 16, line 7, and ending on page 16, line 
20, with the following paragraph: 

—After deriving the mapping in step 112, the higher-resolution source image is 
created as illustrated in steps 1 14-120. To facilitate explanation of the method, a 
"large pixel" is defined as the smallest pixel of the original source image and as a pixel 
of the same size in the principal auxiliary image, which typically must be calculated by 
averaging even smaller pixels present in the principal auxiliary image. Pixels [[54]] 
55 of the re-sampled auxiliary image 38 are large pixels. A "small pixel" is defined as 
the desired pixel size for the enhanced source image . In the example discussion, there 
are four small pixels that cover the same area as one large pixel, but different 
mappings, or numbers of small pixels such as five, nine, sixteen or any other 
convenient number, may be used. The only requirements are that the small pixels of 
the principal auxiliary image match the small pixels of the enhanced source and that an 
integral number of small pixels are covered by one large pixel. In step 114, each large 
pixel of the source image [[34]] 36 is subdivided into small pixels (typically a 2x2 
array). -- 
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Please replace the paragraph beginning on page 17, line 13, and ending on page 17, 
line 20, with the following paragraph: 

—To illustrate a specific embodiment of the method of the present invention, 
the following conditions are assumed. The imaging data applied to the mapping 
function in step 1 10 are the intensities of the average pixels determined in step 104 for 
the source image [[34]] 36 and the auxiliary image [[36]] 34 and the gains between 
pairs 76 of corresponding pixels of the source and resampled auxiliary image 
difference clusters 70. The pixels of the source image 36 are thr ee dim e nsional three- 
dimensional , having red, green and blue intensities. The pixels of the auxiliary image 
34 and resampled auxiliary image 38 are on e dim e nsional one-dimensional , having a 
single panchromatic intensity. - 

Please replace the paragraph beginning on page 1 8, line 1, and ending on page 1 8, line 
14, with the following paragraph: 

—Step 112 may be performed using a clustering technique, such as vector 
quantization. Suppose for example that a three-color image and a co-registered 
panchromatic image are available. Each of the three color planes are to be enhanced, 
one at a time. One is chosen as the source, the panchromatic image is the principal 
auxiliary image, and all three color planes are used as additional auxiliary images. 
According to this embodiment, the image processor 10 assembles a codebook relating 
gain to imaged object as identified by spectral signature. First a conditional gain must 
be calculated for each average pixel determined in step 104 for each color in the 
source image [[34]] 36. To accomplish this, four conditional gains are first calculated 
in steps 106 and 108 based on the 2x2 matrix of small pixels that correspond to the 
average pixel as follows: 
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gain-1 =(upper left comer small pixel (source - average pixel (source))/(upper 
left comer small pixel (principal Auxiliary - average pixel (princ. Aux.)).-- 

Please replace the paragraph beginning on page 19, line 12, and ending on page 20, 

line 5, with the following paragraph: 

—Since there are typically a large number of average pixels in an image, some 
form of clustering is required to represent the characteristics of the imaged materials 
present in the images in a reduced set of intensity vectors. A simple method of vector 
quantization is the following: Take the first four-dimensional intensity vector as a 
cluster center and store it in the first position of the codebook. At each position in the 
codebook also store, for example, the gain vector and the number of pixels (np) 
represented by this cluster center. For every other pixel, compare [[it's]] its four- 
dimensional intensity vector with every stored intensity vector in the codebook. If it is 
close to one of the stored intensity vectors, average its gain vector with the gain vector 
of the stored vector weighing the previously stored gain vector by np, and 
incrementing np by 1 . If it is close to one of the stored intensity vectors, average its 
gain vector with the gain vector of the stored vector weighing the previously stored 
gain vector by np, and incrementing np by 1 . If it is not close, store the intensity 
vector and its associated gain vector as a new entry in the codebook. To determine 
whether or not the intensity vectors are close, a distance measure needs to be selected 
and a threshold set. Typical distance measures are Euclidean, sum of absolute 
differences of the component of the intensity vector, and maximum of the absolute 
differences of the components of the intensity vector.— 

Please replace the paragraph beginning on page 20, line 10, and ending on page 21, 

line 6, with the following paragraph: 

—Rendering of the higher-resolution source image is accomplished as follows: 
Define "large pixel" as the smallest pixel of the original source image and a pixel of 
the same size in the principal auxiliary image, which typically must be calculated by 
averaging smaller pixels present in the principal auxiliary image. Define "small pixel" 
as the desired pixel size of the enhanced source image. Typically, there are four small 
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pixels covering the same area as one large pixel, but different mappings are possible. 
The only requirements are that the small pixels of the principal auxiliary image match 
the small pixels of the enhanced source and that an integral number of small pixels are 
covered by one large pixel. In step 114, each large pixel of the source image [[34]] 36 
is subdivided into small pixels (typically a 2x2 array). In step 1 16, for each small 
pixel of the corresponding large pixel of the principal auxiliary image, compute the 
differences between it and the large pixel intensity (which is the average of all of the 
small pixels it covers). Then, in step 1 18, for each small pixel, retrieve from the [[an]] 
codebook an entry having a four dim e nsional four-dimensional intensity vector to 
which the corresponding large pixel's 4 dim e nsional four-dimensional intensity vector 
is the closest. In step 120, the processor computes the small pixel intensity of the 
enhanced source image as the intensity of the large source pixel + (the associated gain) 
times (the difference between the small pixel and large pixel of the principal auxiliary 
image) for each color. 
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